Deep Decision Trees for Discriminative Dictionary Learning with Adversarial Multi-Agent Trajectories

With the explosion in the availability of spatio-temporal tracking data in modern sports, there is an enormous opportunity to better analyse, learn and predict important events in adversarial group environments. In this paper, we propose a deep decision tree architecture for discriminative dictionary learning from adversarial multi-agent trajectories. We first build up a hierarchy for the tree structure by adding each layer and performing feature weight based clustering in the forward pass. We then fine tune the player role weights using back propagation. The hierarchical architecture ensures the interpretability and the integrity of the group representation. The resulting architecture is a decision tree, with leaf-nodes capturing a dictionary of multi-agent group interactions. Due to the ample volume of data available, we focus on soccer tracking data, although our approach can be used in any adversarial multi-agent domain. We present applications of proposed method for simulating soccer games as well as evaluating and quantifying team strategies.Comment: To appear in 4th International Workshop on Computer Vision in Sports (CVsports) at CVPR 201

Tracking by Prediction: A Deep Generative Model for Mutli-Person localisation and Tracking

Current multi-person localisation and tracking systems have an over reliance on the use of appearance models for target re-identification and almost no approaches employ a complete deep learning solution for both objectives. We present a novel, complete deep learning framework for multi-person localisation and tracking. In this context we first introduce a light weight sequential Generative Adversarial Network architecture for person localisation, which overcomes issues related to occlusions and noisy detections, typically found in a multi person environment. In the proposed tracking framework we build upon recent advances in pedestrian trajectory prediction approaches and propose a novel data association scheme based on predicted trajectories. This removes the need for computationally expensive person re-identification systems based on appearance features and generates human like trajectories with minimal fragmentation. The proposed method is evaluated on multiple public benchmarks including both static and dynamic cameras and is capable of generating outstanding performance, especially among other recently proposed deep neural network based approaches.Comment: To appear in IEEE Winter Conference on Applications of Computer Vision (WACV), 201

A Quantized Interband Topological Index in Two-Dimensional Systems

We introduce a novel gauge-invariant, quantized interband index in two-dimensional (2D) multiband systems. It provides a bulk topological classification of a submanifold of parameter space (e.g., an electron valley in a Brillouin zone), and therefore overcomes difficulties in characterizing topology of submanifolds. We confirm its topological nature by numerically demonstrating a one-to-one correspondence to the valley Chern number in $k\cdot p$ models (e.g., gapped Dirac fermion model), and the first Chern number in lattice models (e.g., Haldane model). Furthermore, we derive a band-resolved topological charge and demonstrate that it can be used to investigate the nature of edge states due to band inversion in valley systems like multilayer graphene.Comment: 6 pages and 3 figures in main text, 2 pages in supplementary materia

Tree Memory Networks for Modelling Long-term Temporal Dependencies

In the domain of sequence modelling, Recurrent Neural Networks (RNN) have been capable of achieving impressive results in a variety of application areas including visual question answering, part-of-speech tagging and machine translation. However this success in modelling short term dependencies has not successfully transitioned to application areas such as trajectory prediction, which require capturing both short term and long term relationships. In this paper, we propose a Tree Memory Network (TMN) for modelling long term and short term relationships in sequence-to-sequence mapping problems. The proposed network architecture is composed of an input module, controller and a memory module. In contrast to related literature, which models the memory as a sequence of historical states, we model the memory as a recursive tree structure. This structure more effectively captures temporal dependencies across both short term and long term sequences using its hierarchical structure. We demonstrate the effectiveness and flexibility of the proposed TMN in two practical problems, aircraft trajectory modelling and pedestrian trajectory modelling in a surveillance setting, and in both cases we outperform the current state-of-the-art. Furthermore, we perform an in depth analysis on the evolution of the memory module content over time and provide visual evidence on how the proposed TMN is able to map both long term and short term relationships efficiently via a hierarchical structure

Metabolic analysis of Developmental progression in Drosophila

The growth and development of all animals involves transitions between different physiological states. The key developmental transition of critical weight (CW) in the fruit fly Drosophila melanogaster dramatically changes the growing larva’s response to nutrient restriction (NR). Developmental progression is arrested by NR before CW whereas it proceeds without delay when NR occurs after CW. It is known that the time of onset of CW and other developmental transitions are regulated by the steroid hormone ecdysone but questions remain concerning the nature of the physiological changes at CW and how they might confer NR-resistant developmental progression. To begin to answer these questions, I have analysed how the larval metabolome changes when nutrition is altered either side of the CW transition. The larval metabolome was recorded via nuclear magnetic resonance (NMR) spectroscopy and fitting reference spectra to recorded peaks enabled identification of the metabolites. Absolute metabolite concentrations could then be back-calculated from these spectra using the volume determination with two standards (VDTS) technique (Ragan, et al. 2013), which was further adapted to measure metabolite concentrations from the volume released from homogenisation of solid whole larval and adult samples. Through use of these techniques, I found that progression past CW correlates with the ability of fed and NR larvae to sustain a substantial increase in the concentration of tyrosine. An interesting interplay between tyrosine and a possible storage form of the metabolite: o-phosophotyrosine (OPT), suggests a process regulating the conversion between the two that may indirectly affect the biosynthesis of ecdysone. Dietary and genetic manipulations have been undertaken to draw a molecular mechanism for how varying tyrosine levels affected by CW attainment can effect time to pupariation (larval maturation). These results highlight how the field of NMR-metabolomics can be used to direct subsequent experiments to address biological questions

A Robust Interpretable Deep Learning Classifier for Heart Anomaly Detection Without Segmentation

Traditionally, abnormal heart sound classification is framed as a three-stage process. The first stage involves segmenting the phonocardiogram to detect fundamental heart sounds; after which features are extracted and classification is performed. Some researchers in the field argue the segmentation step is an unwanted computational burden, whereas others embrace it as a prior step to feature extraction. When comparing accuracies achieved by studies that have segmented heart sounds before analysis with those who have overlooked that step, the question of whether to segment heart sounds before feature extraction is still open. In this study, we explicitly examine the importance of heart sound segmentation as a prior step for heart sound classification, and then seek to apply the obtained insights to propose a robust classifier for abnormal heart sound detection. Furthermore, recognizing the pressing need for explainable Artificial Intelligence (AI) models in the medical domain, we also unveil hidden representations learned by the classifier using model interpretation techniques. Experimental results demonstrate that the segmentation plays an essential role in abnormal heart sound classification. Our new classifier is also shown to be robust, stable and most importantly, explainable, with an accuracy of almost 100% on the widely used PhysioNet dataset

Learning Temporal Strategic Relationships using Generative Adversarial Imitation Learning

This paper presents a novel framework for automatic learning of complex strategies in human decision making. The task that we are interested in is to better facilitate long term planning for complex, multi-step events. We observe temporal relationships at the subtask level of expert demonstrations, and determine the different strategies employed in order to successfully complete a task. To capture the relationship between the subtasks and the overall goal, we utilise two external memory modules, one for capturing dependencies within a single expert demonstration, such as the sequential relationship among different sub tasks, and a global memory module for modelling task level characteristics such as best practice employed by different humans based on their domain expertise. Furthermore, we demonstrate how the hidden state representation of the memory can be used as a reward signal to smooth the state transitions, eradicating subtle changes. We evaluate the effectiveness of the proposed model for an autonomous highway driving application, where we demonstrate its capability to learn different expert policies and outperform state-of-the-art methods. The scope in industrial applications extends to any robotics and automation application which requires learning from complex demonstrations containing series of subtasks.Comment: International Foundation for Autonomous Agents and Multiagent Systems, 201

Physical Adversarial Attacks for Surveillance: A Survey

Modern automated surveillance techniques are heavily reliant on deep learning methods. Despite the superior performance, these learning systems are inherently vulnerable to adversarial attacks - maliciously crafted inputs that are designed to mislead, or trick, models into making incorrect predictions. An adversary can physically change their appearance by wearing adversarial t-shirts, glasses, or hats or by specific behavior, to potentially avoid various forms of detection, tracking and recognition of surveillance systems; and obtain unauthorized access to secure properties and assets. This poses a severe threat to the security and safety of modern surveillance systems. This paper reviews recent attempts and findings in learning and designing physical adversarial attacks for surveillance applications. In particular, we propose a framework to analyze physical adversarial attacks and provide a comprehensive survey of physical adversarial attacks on four key surveillance tasks: detection, identification, tracking, and action recognition under this framework. Furthermore, we review and analyze strategies to defend against the physical adversarial attacks and the methods for evaluating the strengths of the defense. The insights in this paper present an important step in building resilience within surveillance systems to physical adversarial attacks